We'll leave feature caching on for now, since it can speed up authoring. If you need to disable it, you can click the drop-down triangle next to Run to access the Run menu and then deselect Enable Feature Caching.
After completing this lesson, you’ll be able to:
Jennifer has edited her data’s schema using an AttributeManager. She knows the data changed because she can see the green attribute ports showing successful schema mapping. However, Jennifer wants to see the changes to her data in Visual Preview.
Jennifer can do this using feature caching. Feature caching is an authoring mode set on the Run menu and enabled by default. A local data cache is stored at every output port in the workspace when enabled. These caches let you view and compare data anywhere in your workspace.
Feature caching is useful when you are authoring a workspace. It lets you use iterative and incremental development to add one transformer or feature type at a time, create a cache, and inspect it to confirm the data looks as you expect. It is particularly advantageous when working with web, database, or compressed data, allowing you to download, query, or extract the data once and work with a cache, saving time and effort when reading large datasets or making API calls. However, creating these caches takes time, so it’s wise to disable this mode when you want FME to run at peak efficiency.
We'll leave feature caching on for now, since it can speed up authoring. If you need to disable it, you can click the drop-down triangle next to Run to access the Run menu and then deselect Enable Feature Caching.
Jennifer will use another feature, partial runs, to speed up her authoring of workspaces and debug them as she works.
Partial runs allow you to run specified sections of your workspace instead of the entire thing. This feature works in tandem with feature caches to enable incremental development. When you add a new transformer or feature type, you can run that new object independently and inspect its cache for any problems. Authoring workspaces this way saves time by allowing you to detect and fix problems early.
Jennifer continues to work on her workspace in FME Workbench (2024.0 or later).
Jennifer wants to run her workspace to ensure the AttributeManager worked properly. She could just click Run > Rerun Entire Workspace. However, as workspaces grow, running only the changed section is better. She does this by clicking the AttributeManager to select it, then clicking Run To This above the transformer. She can see what part of the workspace will run because it is highlighted in green.
The relevant section of the workspace runs.
She sees caches appear on her BusinessOwners JSON bookmark output port and the AttributeManager transformer. A green magnifying glass icon indicates a feature cache is available to inspect. She clicks it to open that cache in Visual Preview.
As covered in the Connect to Data course, you can also inspect caches by clicking any object with a cache (feature type or transformer) as long as Automatic Inspect on Selection is enabled. Note, however, that this inspects all the caches on that object at once. You’ll have to use the Display Control window and the drop-down menu in Table View to control which cache you are inspecting.
Visual Preview shows the features of the AttributeManager output port. Jennifer looks at Table View to ensure the schema appears as intended. It does:
Partial runs will utilize any connected caches “upstream” (i.e., earlier in the data flow). As few transformers or feature types will run as required to carry out the partial run, saving Jennifer time. For example, if Jennifer had already run the AttributeManager, using Run To This on the writer feature type would use the cache from the AttributeManager.
Now that she’s confirmed the schema is correct, Jennifer wants to write the data, so she hovers over the Run button. Notice how the reader feature type is not highlighted in green. FME will use its cache instead of rereading the data. The rest of the workspace runs when she clicks Run, and FME writes her data.
These examples are pretty simple, since we only have a few objects on the Canvas. But this technique becomes very important as you build bigger workspaces. Use the green highlight to determine which parts of the workspace will run.
⭐ New for FME 2024.2: you can now remove selected feature caches. Right-click them and choose Clear Feature Cache. Clearing the feature cache will also clear the caches of all downstream transformers connected to it.
If Jennifer changed her workspace and wanted to update all the caches, she could choose Rerun Entire Workspace from the Run menu.
This workspace uses an HTTPCaller to get its source data, so just rerunning that transformer will get up-to-date data. However, if Jennifer were reading from a file or database and she wanted to update the data, she could find the reader in the Navigator, right-click it, and select Update. When the underlying data changes, additional features matching the existing reader schema will always be read, even without using Update. But if the schema changes (e.g., new table, new attributes, renamed attributes, etc.), these changes will only be reflected in the workspace if the reader is updated.
Learn more about updating readers and writers
Learn how to make dynamic workspaces that automatically account for schema changes, or see the Advanced Reading and Writing learning path.
Make sure you have followed along with Jennifer’s steps.